Improved Bayesian Posterior Probability...*! Neural Networks in Speech ffittpd^6(2S9.57.104/sear^^ 



This is G o o g I e's cache of http://www.cse.oqi.edu/colloquia/event/81.htiTil . 

Google's cache is the snapshot that we took of the page as we crawled the web. 

The page may have changed since that time. Click here for the current page without highlighting. 

To link to or bookmark this page, use the following url: 

http://www.google.com/search?q=ra^ 

Google is not affiliated with the authors of this page nor responsible for its content. 



These search terms have been highlighted: posterior probability ^ 



BgJ <X( SCXOCL Or SCIENCE fi ENCKNKERING 
Bri OREGON I HEALTH i ' SCIENCE UNIVERSITY 

Department of 
Computer Science 
and Engineering 



Education | Admission | Research | Student Life | People | News | Etc, 



News & Events 

News and Events List 

Reading Groups 

Colloquia 
... upcoming 
... by year 

CSE Employment Opportunities 
20/20 Dept. History 



Improved Bayesian 

[Pijilfiiiiiliiiiii^i 

Estimates with Neural 
Networks in Speech 
Recognition 

Wednesday, May 14th, 1997 at 10:00 am 
Jack Murdock Laboratory, Room 561 



Wei Wei 



Abstract 



Neural networks can be used as 



w^ian post erior probability 

estimators in speech recognition system 
when we have arbitrarily large amount of 
training data and arbitrarily complex 
networks that are perfectly trained. 
However, in practice, these assumptions 
are not satified and therefore affect 
system's performance negatively. We use 
empirical estimates based on histograms 
to measure how good neural network 
probability estimates are for each class. 
The deviations shown by histograms 
confirm that neural networks fall short of 
being ideal estimators. We address this 
problem and propose a new method by 
selecting well-trained classes and 
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remapping their neural network outputs 
based on the histograms to match the 
data and thereby improve the estimates 
of posterior probabilities. Experiments on 
OGI Census Year corpus resulted in a 
27% reduction at sentence-level error 
(from 6.3% to 4.6%); experiments on 
OGI Digit corpus resulted in a 25.4% 
reduction at word-level error (from 
6.03% to 4.5%) and 20.8% reduction at 
sentence-level error (from 19.80% to 
15.68%), with negligible amount of 
parameters added for remapping and 
negligible calculation used for remapping. 
The results of McNemar's test showed the 
statistical significance of this new 
approach. 
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Neural network outputs may fail to 
estimate posterior probabilities well 
especially for infrequent classes that are 
short of training samples. Because 
remapping as a post-processing depends 
on neural networks, we need to improve 
neural networks as g^t^ 
pro bability estimators fundamentally. 
My ongoing work towards Ph.D. thesis 
includes: (1). Improving neural networks 
as m^mmim>b£M$m estimators by 
developing new activity functions or new 
cost functions;(2). Studying speech 
features to develop 
speech-recognition-oriented neural 
networks;(3). Simulations on artificial 
data (toy problems) to study the 
mechnism of curre nt neural networks as 

estimators under a controllable 
environment and to develop new neural 
network models. Preliminary work has 
been done in these areas, with promising 
theoretical and experimental results 
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